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Abstract 

Stochastic  Resorumce.  is  a  nor, linear  stotl^stic  phenom¬ 
enon  which  can  cause  a  transfer  cf  energy  from  a  random 
process  (noise)  to  a  periodic  signal  over  a  certain  range  of 
signal  and  system  parameters.  It  has  been  observed  in 
many  diverse  natural  and  physical  systems,  and  may  be  one 
means  by  which  biological  sensor  systems  amplify  weak 
sensory  signals  for  detection.  This  paper  is  an  evaluation 
of  the  use  of  the  Stochastic  Resonance  phenomenon  as  a 
tool  for  signal  processing  in  terms  of  its  processing  gain. 

1:  Introduction 

Stochastic  Resonance  is  a  nonlinear  phenomenon  which 
has  been  shown  to  occur  in  bi-stable  systems  or  In  two  state 
devices.  Over  a  certain  range  of  signal  and  system  parame¬ 
ters  it  can  cause  what  amounts  to  a  transfer  of  energy  from 
a  random  process  (noise)  to  a  periodic  process  (signal). 
This  has  the  potential  to  dramatically  improve  the  signal  to 
noise  ratio  at  the  output  of  the  device.  Use  of  a  single  non¬ 
linear  two  state  device  will  be  shown  to  be  lossy  as 
compared  to  the  use  of  the  power  spectrum  for  detecting  a 
sinusoid  in  white  noise.  However,  globally  coupling  many 
two  state  devices  appears  to  have  the  potential  to  overcome 
these  losses,  and  possibly  exceed  the  gains  achievable  by 
linear  systems  due  to  an  enhancement  of  the  Stochastic 
Resonance  effect  over  what  might  be  expected  in  a  single 
two  state  device. 

The  “classical”  description  of  the  phenomenon  is  that  of 
a  particle  in  a  dual  well  potential  which  is  excited  by  a 
strong  stochastic  process  and  a  weak  periodic  process. 
Here,  weak  means  that  the  force  applied  by  the  periodic  ex¬ 
citation  alone  is  not  sufficient  to  overcome  the  barrier,  so 
no  state  transitions  occur  in  the  absence  of  stochastic  forc¬ 
ing.  When  noise  of  appropriate  strength  is  added  to  the  pe¬ 
riodic  forcing,  state  transitions  which  have  a  random  com¬ 
ponent  and  a  periodic  component  occur.  The  periodic 
component  of  the  transitions  (output)  contains  the  funda¬ 
mental  frequency  and  sometimes  odd  harmonics  of  the  pe¬ 
riodic  excitation.  As  one  increases  the  strength  of  the  ran¬ 


dom  forcing,  the  component  of  the  power  spectrum  of  the 
transitions  which  is  at  the  frequency  of  the  periodic  forcing 
increases  to  a  peak  value  for  a  critical  value  of  noise 
strength,  and  then  decreases  with  further  increases  of  noise 
strength  until  the  stochastic  forcing  completely  dominates 
the  state  transitions.  It  is  this  peaking  that  inspired  the  term 
Stochastic  Resonance.  There  is  also  a  peaking  of  the  signal 
to  noise  ratio  of  the  periodic  component  of  the  transitions, 
occurring  near  the  amplitude  peak. 

The  dual  well  model  (Fig.  1)  is  the  one  most  commonly 
found  in  the  literature.  An  equivalent  description  which  is 
better  suited  to  the  purposes  of  this  paper  is  an  input/output 
or  transfer  characteristic.  Figure  2  shows  two  examples  that 
should  be  familiar  to  the  reader  which  support  Stochastic 
Resonance;  that  of  a  two  state  magnetic  device,  and  of  a 
Schmitt  Trigger.  Here  the  Stochastic  Resonance  phenome¬ 
non  may  be  viewed  as  the  response  of  a  two  state  device 
driven  by  periodic  and  a  stochastic  signal,  wherein  the  peri¬ 
odic  signal  alternately  increases  the  probability  of  a  transi¬ 
tion  from  one  state  to  the  other,  with  the  noise  facilitating 
the  transitions. 


Figure  1.  Particle  in  a  Dual  Well 


out 


in 


Schmitt  Trigger 


Figure  2.  Transfer  Characteristics 
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2:  Background 

Stochastic  Resonance  was  first  reported  and  analyzed  in 

[I]  as  an  Interesting  effect  in  nonlinear  dynamics.  It  was 
soon  after  proposed  as  a  possible  explanation  for  the  ob¬ 
served  periodic  occurrences  of  the  earth’s  ice  ages  in  [2]  and 
[3].  For  this  case,  the  two  states  of  the  system  are  taken  to 
be  the  earth’s  climate  as  it  is  at  present  and  as  it  is  during 
an  ice  age.  The  “noise”  of  the  system  is  the  variation  in  the 
absorption  or  reflection  of  incident  solar  energy  caused  by 
daily  and  seasonal  weather  patterns,  cloud  cover,  ice  cover, 
etc.  The  weak  periodic  force  is  the  variation  of  incident  so¬ 
lar  energy  caused  by  a  small  periodic  eccentricity  in  the 
earth’s  orbit  having  a  period  of  100,000  years. 

An  experimental  measurement  of  the  behavior  of  a  two 
state  electronic  device,  the  Schmitt  Trigger,  driven  by  a 
weak  periodic  signal  in  noise,  was  reported  in  [4].  The  Sto¬ 
chastic  Resonance  phenomenon  was  observed,  and  the 
location  of  the  resonance  peak  was  shown  to  be  in  the  vicin¬ 
ity  of  where  the  noise  intensity  had  a  Kramers  rate  [5]  close 
to  half  of  the  frequency  of  the  periodic  signal.  The  Kramers 
rate  (in  the  absence  of  signal)  is  given  by: 

r  =  2~,/lU”(0)l  U”(c)  exp(- (1) 

where  U”(0)  and  U”(c)  are  the  curvatures  of  the  potential 
at  the  barrier  and  in  the  wells,  respectively,  AU  is  the  barrier 
height,  and  a  is  the  standard  deviation  of  the  noise.  White 
noise  is  assumed  here,  and  throughout  this  paper. 

Stochastic  Resonance  has  been  observed  in  other  physi¬ 
cal  systems  such  as  ring  lasers  [61,  bi-stable  electron  para¬ 
magnetic  resonance  systems  [7],  magnetoelastic  ribbons 
[8],  and  simulated  in  bi-stable  Superconducting  Quantum 
Interference  Devices  (SQUIDs)  [9],  References  [10]  and 

[II]  contain  a  theoretical  treatment  of  the  Stochastic  Reso¬ 
nance  phenomenon  based  on  the  so-called  adiabatic  as¬ 
sumption,  and  [7]  reported  the  observation  of  a  phase  shift 
occurring  in  the  output  of  the  periodic  component  as  it 
passed  through  the  output  signal  to  noise  ratio  peak  that  is 
characteristic  of  Stochastic  Resonance.  This  gives  added 
validity  to  the  use  of  the  term  resonance,  as  similar  phase 
shifts  are  observed  in  system  with  deterministic  reso¬ 
nances.  A  treatment  based  on  linear  response  theory  is  giv¬ 
en  in  [12]. 

It  has  been  speculated  thru  Stochastic  Resonance  may  be 
exploited  by  biological  sensor  systems  as  a  means  of  in¬ 
creasing  the  signal  to  noise  ratio  of  sensory  inputs  [13]. 
Though  it  has  yet  to  be  demonstrated  in  any  known  biologi¬ 
cal  system,  the  elements  are  there  for  this  to  be  possible. 
Sensory  neurons  are  viewed  as  two  state  systems  (firing  or 
not  firing),  and  noise  induced  switching  has  been  observed 
in  biological  systems:  [14],  [15],  and  [16].  Stochastic  Reso¬ 
nance  can  be  induced  in  artificial  neural  networks  [17],  and 
if  arrays  of  bi-stable  elements  are  globally  coupled,  there 


are  cooperative  effects  which  enhance  the  phenomenon 
[18]. 

Good  surveys  on  the  subject  of  Stochastic  Resonance 
with  extensive  reference  lists  can  be  found  in  [  1 3],  [  19],  and 
[20], 

3:  Stochastic  Resonance  as  a  Signal  Proces¬ 
sing  Tool 

The  question  to  be  addressed  here  is  whether  or  not  it  is 
useful  (or  desirable)  to  incorporate  a  “stochastic  resonator” 
in  a  signal  processing  system  to  aid  signal  detection.  It  is 
well  known  that  the  linear  filter  which  maximizes  output 
signal  to  noise  ratio  is  the  “matched  filter”  [21],  [22],  [23]. 
There  is  no  such  generalized  theory  for  the  class  of  nonlin¬ 
ear  filters  so  it  is  possible  (though  not  yet  proven  or  dis¬ 
proved)  that  a  nonlinear  filter  or  operation  could  create  a 
greater  output  signal  to  noise  ratio. 

The  Schmitt  Trigger  is  the  “stochastic  resonator”  device 
chosen  for  the  evaluation  here.  It  is  simple  to  implement 
in  hardware  or  software,  it  reduces  dynamics  to  just  switch¬ 
ing  events,  and  is  a  constant  output  energy  device.  Thus  it 
is  easy  to  show  that  if  the  output  signal  to  noise  ratio  in¬ 
creases  with  increasing  input  noise  intensity,  the  increase  in 
output  signal  power  is  at  the  expense  of  noise  power.  A 
computer  simulation  consisting  of  a  sinusoid  in  “white” 
noise  driving  a  Schmitt  Trigger  was  used  for  the  evaluation. 
Adjustable  parameters  were  signal  amplitude,  noise  vari¬ 
ance,  and  threshold  of  the  Schmitt  Trigger.  The  power 
spectra  of  the  Schmitt  Trigger  input  and  output  were  used 
to  measure  the  input  and  output  signal  to  noise  ratios.  Pow¬ 
er  spectral  estimation  via  Fourier  methods  is  commonly 
used  for  detecting  sinusoids  of  unknown  frequency  and 
phase  in  noise,  and  is  the  closest  approximation  to  the 
matched  filter  achievable  for  that  case  with  Gaussian  noise 
[22],  [23],  when  the  binwidth  [24]  is  matched  to  the  band¬ 
width  of  the  unknown  signal.  For  these  reasons  the  power 
spectral  estimate  was  chosen  to  be  the  baseline  for  compari¬ 
son  in  this  paper. 

Figure  3  is  a  plot  of  typical  input  and  output  power  spec¬ 
tra  for  the  simulations.  Figures  4  and  5  are  averages  of  the 
signal  to  noise  ratios  at  the  input  and  output  of  the  Schmitt 
Trigger  measured  from  many  power  spectra  for  a  variety  of 
cases  with  different  signal  amplitudes  ana  input  noise  pow¬ 
er.  Note  that  for  a  single  Schmitt  Trigger,  output  signal  to 
noise  ratio  was  always  less  than  that  of  the  input,  but  was 
within  2  dB  for  large  noise  variance.  Thus  the  Stochastic 
Resonance  process  of  a  single  device  is  lossy  as  compared 
to  standard  spectral  estimation  techniques. 

4:  Output  Statistics,  Non-Poissonian  Behav¬ 
ior  at  High  Noise  Levels 

A  Schmitt  Trigger  driven  by  zero  mean  noise  will  have, 
under  appropriate  conditions,  switching  events  that  are 
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Figure  4.  Input  and  Output  Signal  to  Noise  Ratio 
with  Signal  Amplitude  .5,  Threshold  1 


Figure  5.  Input  and  Output  Signal  to  Noise  Ratio 
with  Signal  Amplitude  .1,  Threshold  1 


Poisson  distributed  [25].  That  is,  the  probability  of  k 
switching  events  over  some  time  interval  t  will  be: 


P(k:t) 


(2) 


where  X  is  the  “rate”  of  the  Poisson  process.  Then  the  auto¬ 
correlation  function  of  the  Schmitt  Trigger  output  will  be 
[25]: 


R(t)  =  exp[- 2Xx]  (3) 

and  its  power  spectrum  will  be: 

S<“)-4V^?  <4> 

Note  that  the  rate,  X ,  can  be  estimated  from  the  power  spec¬ 
tral  estimate  of  the  Schmitt  Trigger  output  since  (from  (4)): 


^  —  2*  —  (5) 

It  is  noted  here  that  the  Poisson  Distribution  is  in  fact  a  lim¬ 
iting  case  of  Bernoulli  Trials: 


PfA  occurs  k  times) 


pkq 


n— k 


(6) 


where  n  is  the  number  of  trials,  p  is  the  probability  of  occur¬ 
rence  of  event  A,  and  q  =  1  -  p.  Following  the  Poisson 
Theorem,  one  can  approximate  (6)  by  (2)  for  a  continuous 
time  process  as  n-*  <» ,  t  -*  0,  and  as  n/t  Fora  discrete 

time  process  one  can  approximate  (6)  by: 


P(A  occurs  k  times) 


(7) 


asn-*  o°  ,andp-*0.  Using  (5),  (7),  and  (2),  one  can  check 
the  validity  of  simulation  results,  or  as  will  be  shown,  iden¬ 
tify  the  point  where  there  is  a  departure  from  Poisson  be¬ 
havior  due  to  a  violation  of  the  assumptions  of  the  Poisson 
Theorem. 


If  the  noise  driving  the  Schmitt  Trigger  is  zero  mean  and 
Gaussian,  the  probability,  p,  of  a  switching  event  occurring 
at  any  instant  in  time  is: 

p  =  erfc(Th/o)  (8) 

where  Th  is  the  switching  thresiiold  of  the  Schmitt  Trigger, 
and  o  is  the  standard  deviation  of  the  noise,  with  erfc(x)  de¬ 
fined  as  in  [22], 

Table  1  is  a  comparison  of  theoretical  vs.  experimental 
(simulated)  determination  of  the  Poisson  rate,  X  ,  for  a 
Schmitt  Trigger  with  switching  thresh  :!is  at  >7-  1,  aad  th" 
input  noise  variances  shown.  Note  that  the  experimental 
rale  is  higher  than  the  predicted  value  for  input  noise  vari¬ 
ance  greater  than  2,  or  for  probability  of  a  switching  event 
greater  than  .2,  due,  no  doubt,  to  a  violation  of  Poisson 
Theorem  assumptions. 
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Theoretical 


Experimental 


above  the  threshold  of  detection  for  the  fine  case.  An  ex¬ 
haustive  study  or  analysis  of  this  was  not  done. 


P  = 

Variance 

erfc(Th/o) 

np 

X 

*3dB 

0.25 

.023 

47 

45 

14 

0.5 

.08 

163 

151 

48 

1.0 

.159 

327 

332 

105 

2.0 

.24 

491 

622 

198 

4.0 

.39 

634 

854 

272 

Table  1.  Theoretical  vs.  Experimental  Determination  of  X 
(note:  Th  =  1.  n  =  2048) 

5:  Dithering,  Stochastic  resonance,  Coarse 
Quantization 

Dithering  is  a  technique  used  to  ameliorate  undesirable 
nonlinearities  in  a  system.  It  consists  of  adding  a  small 
amount  of  (white)  noise  to  a  system.  In  digital  signal  pro¬ 
cessing,  adding  noise  to  the  least  significant  bit  of  a  quan¬ 
tized  signal  is  often  used  to  reduce  the  nonlinear  affects  of 
the  quantization.  It  has  been  reported  that  dithering  can  al¬ 
low  the  detection  of  signals  smaller  in  amplitude  than  the 
least  significant  bit  [26],  or,  for  image  processing,  allow  the 
detection  of  image  features  less  intense  than  the  lowest 
quantization  level. 

It  was  suggested  in  [26]  that  the  ability  to  detect  these  un¬ 
der-resolved  signals  was  due  to  a  “rounding”  of  the  comers 
of  the  quantization  steps  which  would  create  a  linearization 
of  the  stepped  transfer  characteristic  from  input  to  output  of 
the  quantization  process.  This  could  be  the  case  if  the  noise 
were  multiplicative,  however  this  is  not  possible  with  a  lin¬ 
ear  summation  of  noise.  A  much  more  plausible  explana¬ 
tion  is  that  there  is  a  mechanism  similar  to  Stochastic  reso¬ 
nance  which  allows  this  detection  of  under-resolved 
signals.  The  diagram  in  Figure  6  illustrates  this  mecha¬ 
nism:  an  under-resolved  sinusoid  added  to  noise  of  ap¬ 
propriate  intensity  will  alternately  increase  the  probability 
of  transition  to  a  higher  or  lower  quantization  state  at  its 
high  and  low  peaks,  and  otherwise  tend  to  stay  in  the  current 
quantization  state. 

Continuing  this  line  of  reasoning,  a  few  simulations  were 
run  to  compare  the  processing  gain  (loss)  of  coarse  quanti¬ 
zation  to  that  of  a  Schmitt  Trigger.  Note  that  the  key  differ¬ 
ence  between  the  transfer  characteristics  of  the  two  is  that 
the  Schmitt  Trigger  lias  hystcreri'-  while  the  “staircase” 
characteristic  of  die  quantizer  is  closer  to  being  linear.  The 
results  of  this  cursory  test  indicated  that  the  coarse  quantiz¬ 
er  outperformed  the  Schmitt  Trigger,  and  matched  the  per¬ 
formance  of  a  finely  quantized  simulation  until  a  point  just 


Figure  6.  Quantization  and  Dithering 

6:  Coupled  Bi-Stable  Devices 

A  study  of  the  influence  of  a  “bath”  of  overdamped  bi¬ 
stable  oscillators  coupled  to  a  single  bi-stable  oscillator  can 
be  found  in  [18],  with  a  more  detailed  analysis  forthcoming 
in  [27],  Equation  (9)  is  the  model  for  the  oscillators,  cou¬ 
pling,  and  periodic  plus  noisy  forcing  that  was  considered: 

N 

CjUj  =  V  i^tanh  U:  -  ^  +  F,{t)  +  qsinM  (9) 

U  Ki 

where  the  summation  term  defines  the  coupling,  and  the 
last  two  terms  are  the  noisy  and  periodic  forcing  respective¬ 
ly.  The  variables  Ui  denote  the  state  variables  (analogous  to 
the  membrane  potentials  in  a  biological  neuron  applica¬ 
tion)  of  each  element,  and  Rj  and  C;  represent  the  input  ca¬ 
pacitance  and  transmembrane  resistance  of  each. 

If  one  assumes  the  time  constant  of  the  element  of  inter¬ 
est  to  be  much  longer  than  that  of  the  others,  ie.: 

R,C,  >  >  RA  for  i  >  1  (10) 

and  if  the  frequency  of  the  periodic  forcing,  w,  is 
constrained  to  be  less  than  the  Kramers  rate  of  the  system 
when  driven  only  by  the  noise,  then  one  can  adiabatically 
eliminate  the  “bath”  variables  (uj ,  for  i  >  1)  from  Equation 
(9)  to  yield  a  reduced  model  of  the  following  form  [18], 
[28]: 

U)  =  -au,  +  ptanhu,  +  6  sin  cot  +  oJF(t)  (11) 

Simulations  of  this  reduced  model  have  been  done,  and  en¬ 
hancements  of  output  signal  to  noise  ratio  (as  compared  to 
what  is  observed  with  a  single  oscillator)  have  been 
achieved.  The  behavior  of  the  system  is  very  sensitive  to 
the  choice  of  system  parameters  (coupling,  time  constants, 
etc.)  and  signal  parameters  (input  noise,  bath  noise,  etc.). 
Investigations  into  the  applicability  of  this  phenomenon  to 
signal  processing  are  continuing,  including  the  generation 
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of  a  simulation  of  the  full  dynamics  represented  by  Equa¬ 
tion  (9). 

7:  Conclusion 

Compared  to  standard  power  spectral  estimation  tech¬ 
niques,  the  use  of  a  single  “stochastic  resonator”  device  will 
result  in  a  processing  loss  of  at  least  2  dB  in  the  region  of 
signal  to  noise  ratio  where  the  “resonator”  performs  best 
In  addition,  there  does  not  appear  to  be  any  advantage  to  uti¬ 
lizing  a  single  “resonator”  device  where  signals  are  under¬ 
resolved  due  to  coarse  quantization.  There  is,  however,  evi¬ 
dence  which  suggests  that  use  of  an  array  of  globally 
coupled  devices  may  be  able  to  overcome  the  processing 
loss  of  a  single  device,  and  perhaps  exceed  the  gains  achiev¬ 
able  with  linear  signal  processing  systems. 
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